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ABSTRACT 


New  kinds  of  tests  are  being  evaluated  as  potential 
additions  to  the  Anned  Services  Vocational  Aptitude 
Battery  (ASVAB).  They  are  compared  on  the  basis  of 
die  criterion  variance  they  explain  when  added  to  the 
ASVAB.  The  evaluation  may  use  scores  on  the  ASVAB 
given  during  enlistment  processing,  or  a  new  ASVAB 
may  be  administered  concurrently  widi  the  new  tests. 
This  paper  compares  these  two  research  designs  in  terms 
of  their  effect  on  evaluation  of  new  tests.  The  analysis 
uses  Infimtiy  data  horn  the  Marine  Corps'  Job  Per¬ 
formance  Measurement  {noject,  in  which  concurrent  as 
well  as  enlistment  ASVAB  scores  are  available.  While 
lower  increments  in  ex^dained  variances  are  obtained 
when  the  ASVAB  is  concurrent,  die  difference  between 
die  administratitnis  is  small  compared  to  variations 
across  criterion  variables  and  occupatkmal  areas. 
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EXECimVE  SUMMARY 


INTRODUCTION 

The  Aimed  Services  Vocadimal  Aptitude  Battery  (ASVAB)  is  used  in  selectitm  and 
classification  of  enlisted  personneL  It  coiuains  ten  subtests,  which  measure  four  aptitudes — 
Verbal,  Math,  Speed,  and  TechnicaL  The  services  have  developed  new  tests  for  measuring  other 
traits,  such  as  psychomotor  ability  and  spatial  peiceptitm,  whose  measurement  could  also  help  in 
selectitm  and  classification.  These  tests  are  usefiil  to  die  e^dent  that  they  increase  die  predictive 
power  of  the  ASVAB.  The  purpose  of  the  Enhanced  Computer  Assisted  Testing  ^CAT)  project 
is  to  evaluate  some  new  tests  and  determine  whedier  they  should  be  implemented  nationwide. 
The  ECAT  validation  study  is  expected  d)  begin  by  April  1990. 

Estimation  of  die  increase  in  predictive  power  requires  that  scores  tm  the  ASVAB,  on  the 
new  test,  and  on  a  criterion  be  available  for  a  group  of  recruits.  (The  criterion  variable  measures 
job  performance  or  proficiency.)  Each  recruit  has  taken  the  ASVAB  prior  to  enlistment  The 
criterion  measure  is  closer  in  time  to  the  new  test  than  to  die  ASVAB.  As  a  result  the  i»edictive 
power  of  die  ASVAB  may  be  diluted,  and  die  new  test  may  appear  more  useful  than  it  really  is. 
Such  a  Was  in  the  evaluation  should  be  avoided  as  far  as  possiUe— pertiaps  by  administeiing  die 
ASVAB  again,  concurrently  widi  the  new  test  However,  this  would  require  three  more  hours  of 
testing  time.  Also,  the  recruit  might  not  put  as  much  ei^it  into  taking  an  ASVAB  administered 
purely  for  research  as  into  taking  die  enlistment  test 

OBJECTIVE  AND  DATA 

The  objective  of  this  paper  is  to  determine  whether  enlistment  and  concurrent  ASVAB 
admiidstiatioiis  yield  roughly  equal  values  for  the  increase  in  predictive  power  provided  by  a 
new  test  The  data  set  was  obtained  from  the  Marine  Corps’  Job  Peifonnance  Measuremoit 
(JPM)  project  For  each  Marine  it  ctmtains  hands^  and  job  knowledge  test  scores,  enlistment 
and  concurrent  ASVAB  scores,  and  scores  on  four  new  tests.  To  motivate  examinees  to  put  as 
much  effiait  into  die  concurrent  ASVAB  as  they  had  put  into  die  enlistment  test  the  scores 
became  scores  of  record  if  they  exceeded  previous  ones  by  a  i»especified  amount 

The  hands-on  scores  are  based  tm  job  tasks  iqnesentative  of  job  requirements  in  four 
mflitaiy  oociqpational  qwdalties  (MOSs)  in  die  Infantry:  Rifleman,  Maddnegunner,  Moitaimait 
and  Assanltman.  The  job  kmudedge  teats  were  paper^pendl  tests  of  information  needed  on  the 
job.  The  new  tests  were  Video  Bring  (a  commercial  video  game),  and  paper-pencil  tests  of 
^Mce  Peioqition,  Reasoning,  and  Assembling  Objects. 


RESULTS  AND  CONCLUSIONS 

On  the  whole,  the  new  tests  iQ)peaied  less  useful  whoi  they  were  added  to  concurrent 
ASVA6  dian  to  enlistment  ASVAB.  However,  the  difference  was  small  compared  to  variations 
across  MOSs,  and  between  hands-on  and  job  knowledge  measures  of  job  proficioicy.  Thus,  in 
future  research,  concurrent  administration  of  the  ASVAB  may  be  useftil  if  some  incentive  is 
provided  to  motivate  the  examinees.  Without  such  an  incentive,  the  concurrent  ASVAB  prob¬ 
ably  will  rut  be  worth  the  extra  testing  time  and  expense. 

The  ECAT  validadon  study  of  new  computerized  predictors  includes  a  wide  variety  of  jobs 
and  of  criterion  variables.  Its  results  may  well  show  as  much  variability  across  jobs  and  criteria 
as  in  the  present  research.  If  they  do.  it  win  be  very  difRcult  to  summarize  the  results,  cmnpare 
one  test  with  another,  and  decide  whether  it  is  worthwhile  to  implemem  new  tests  operationally. 
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INTRODUCTION 


The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  is  used  to  select  and  classify 
enlisted  personnel.  It  contains  ten  subtests:  General  Science  (GS),  Arithmetic  Reasoning  (AR), 
Word  Knowledge  (WK).  Paragraph  Comprehension  (PC),  Numerical  Operations  (NO),  Coding 
Speed  (CS),  Auto  and  Shop  Information  (AS),  Math  Knowledge  (MK),  Mechanical  Compre¬ 
hension  (MC),  and  Electronics  Information  (El).  Factor  analysis  of  the  ASVAB  shows  that  these 
subtests  measure  four  factors:  Verbal  (GS,  WK,  and  PQ,  Math  (AR  aiKl  MK),  Speed  (NO  arid 
CS),  and  Technical  (AS,  MC,  and  El)  [1]. 

There  are  traits  the  present  ASVAB  does  not  measure,  such  as  psychomotor  ability  and 
spatial  perception.  The  services  have  developed  new  tests  for  measuring  such  traits.  Addition  of 
new  tests  to  the  ASVAB  is  useful  to  the  extent  that  these  tests  increase  the  predictive  power  of 
the  ASVAB.  Such  increase  is  called  “incremental  validity.”  An  Enhanced  Computer  Assisted 
Testing  (ECAT)  project  is  now  in  progress,  to  evaluate  the  incremental  validities  of  new  tests  and 
to  decide  whether  they  should  be  implemented  operationally.  An  ECAT  validation  study  is 
expected  to  begin  data  collection  by  April  1990. 

Evaluation  of  incremental  validity  requires  recruits  to  be  administered  a  criterion  measure 
of  job  performance  or  proficierKy.  The  data  set  must  contain  scores  on  the  ASVAB,  on  the 
new  test,  and  on  the  criterion  for  a  group  of  recruits.  First  the  criterion  score  is  predicted  using 
the  ASVAB,  and  then  the  new  test  is  added  to  the  regression  equation.  The  increase  in  the 
multiple  correlation  is  the  incremental  validity  of  the  new  test 

Each  recruit  has  taken  the  ASVAB  prior  to  enlistment  In  the  ECAT  validation  study,  the 
new  test  and  the  criterion  will  be  administered  after  the  recruit  has  been  in  a  service  for  some 
time.  Thus  the  criterion  is  closer  in  time  to  the  new  test  than  to  the  ASVAB,  which  may  tend  to 
reduce  the  predictive  power  of  the  ASVAB  and  hence  inflate  the  incremental  validity  of  the  new 
test.  This  bias  can  be  eliminated  by  administering  the  ASVAB  again,  along  with  the  new  test. 
This  “concurrent”  ASVAB  has  two  shortcomings.  One  is  that  three  hours  of  extra  testing  are 
required.  The  other  problem  is  that  the  recruit  may  put  less  effort  into  it  than  into  the  enlistment 
ASVAB  because  it  has  tx)  effect  on  his  future.  In  such  a  case,  the  concurrent  scores  may  have 
less  predictive  validity  than  those  obtained  under  proper  motivation.  Therefore,  it  is  important  to 
know  whether  use  of  enlistment  ASVAB  does  appreciably  overestimate  the  incremental  validity 
of  a  new  test.  If  it  does  not,  a  concurrent  ASVAB  is  unnecessary  and  its  omission  from  the 
ECAT  study  is  technically  justified. 

A  data  set  from  the  Marine  Corps’  Job  Performance  Measurement  (JPM)  project  can  be 
used  to  compare  incremental  validities  obtained  with  enlistment  and  concurrent  ASVAB  scores. 
The  Marine  Corps  has  developed  hands-on  performance  tests  (HOPTs)  in  the  joint-service  JPM 
project  An  oversight  committee  of  the  National  Academy  of  Sciences  has  referred  to  HOPTs  as 
the  “benchmark  measure”  of  job  performance  ([2],  p.  95).  In  addition  to  hands-on  and  job 
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knowledge  tests,  a  concunent  ASVAB  and  some  new  tests  were  administered  to  Marines  in 
four  military  occupational  specialties  in  the  Infantry;  Rifleman  (0311),  Machinegunner  (0331), 
Mortaman  (0341),  and  Assaultman  (0351).  To  motivate  the  examinees,  the  concurrent  scores 
replaced  the  enlistment  scores  in  the  Marine’s  record  if  the  new  scores  exceeded  the  old  ones  by  a 
certain  amount  The  purpose  of  this  paper  is  to  use  the  JPM  data  set  to  compare  enlistment  and 
concurrent  ASVAB  scores  in  terms  of  the  incremental  validities  they  yield  for  four  new  predictors. 

HANDS-ON  PERFORMANCE  MEASURES 

Official  Marine  Corps  publications  and  training  materials,  supplement'd  by  extensive 
job  analyses,  were  used  to  specify  the  domain  of  Infantry  job  requirements.  Jo  a  task  domains 
were  developed  for  each  MOS.  A  common  core  of  infantry  tasks  required  in  all  MOSs  was 
identified  (e.g.,  land  navigation,  tactical  measures,  first  aid,  grenade  launcher),  in  addition  to 
MOS-specific  requirements  (e.g..  Rifleman,  M16A2  rifle;  Machinegunner,  M60  machinegun; 
Mortarman,  bO-mm  and  81-mm  mortars;  and  Assaultman,  Dragon  and  SMAW).  Tasks  were 
sampled  from  each  domain  so  that  hands-on  test  scores  would  generalize  to  the  full  range  of 
Infantry  job  requirements  within  that  domairu  Scores  were  computed  for  both  the  common  core 
and  the  MOS-specific  components  and  then  weighted  to  create  a  hands-on  total  score  for  each 
MOS.  Details  are  provided  in  [3]. 

Because  of  the  uniqueness  of  the  MOS-specific  components,  HOPT  scores  for  the  different 
MOSs  were  not  on  the  same  scale.  To  achieve  comparability,  the  HOPTs  were  linearly 
transformed  to  a  standard  scale  as  follows:  Each  HOPT  was  regressed  on  time  in  service  (TIS), 
its  square  (TIS_SQ)  and  all  ten  enlistment  ASVAB  subtests,  separately  for  each  MOS.  Using  the 
regression  in  a  given  MOS,  the  expeaed  HOPT  score  at  TIS  of  24  months  was  computed  for 
each  person  in  the  1980  Reference  Population  [4].  A  random  normal  error  term  with  a  standard 
deviation  equal  to  the  standard  error  of  estimate  was  added  to  generate  a  simulated  HOPT  score 
for  each  person.  The  mean  and  standard  deviation  of  the  simulated  HOPT  scores  were  calculated 
over  the  Reference  Population.  Then,  using  this  mean  and  standard  deviation,  the  HOPT  scale 
was  transformed  so  that  its  mean  and  standard  deviation  in  the  Reference  Population  were  50  and 
10  (except  for  the  sampling  error  caused  by  the  random  error  term).  Because  the  population 
variance  is  100,  incremental  variances  explained  by  new  tests  are  directly  interpretable  as 
percentages  of  the  total  variance. 

OTHER  TESTS 

Paper-pencil  job  knowledge  tests  (JKTs),  which  asked  for  information  about  tasks  in  the 
HOPT,  were  also  administered.  JKT  scores  wer  standardized  in  the  same  way  as  HOPT  scores. 

Four  new  predictors  were  evaluated:  Video  Firing,  Space  Perception,  Reasoning,  and 
Assembling  Objects.  Video  Firing  assessed  psychomotor  ability  with  a  computer  game  of  firing 
at  a  target  on  a  television  screen.  The  other  three  tests  were  paper-pencil.  Space  Perception 
tested  spatial  visualization  with  items  that  involved  drawings  of  folded  and  unfolded  pieces  of 
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paper.  The  Reasoning  test  measured  spatial  reasoning  and  pattern  recognition.  The  Assembling 
Objects  test  measured  spatial  visualization  and  mental  rotatioa 

SUBJECTS 

The  subjects  were  active-duty  male  Marines  with  various  durations  of  service  in  four  In¬ 
fantry  MOSs:  Rifleman  (0311),  Machinegunner  (0331),  Mortarman  (0341),  and  Assault- 
man  (0351).  The  data  were  collected  as  part  of  the  Marine  Corps’  Job  Performance  Measure¬ 
ment  project  Subjects  were  administered  the  ASVAB  during  the  study.  This  will  be  referred  to 
as  the  concurrent  ASVAB.  In  addition,  enlistment  ASVAB  scores  were  available  from  the 
subjects’  files.  Only  those  Marines  with  complete  data  (i.e.,  criterion,  new  predictor,  and  ASVAB 
scores),  and  no  more  than  48  months  of  service,  were  included  in  the  analyses.  The  useful  sample 
consisted  of  864  Riflemen,  234  Machinegunners,  223  Mortannen,  and  251  Assaultmea 

METHODOLOGY 

Preliminary  analyses  showed  that  results  were  practically  the  same  whether  the  sub¬ 
test  scores  or  faaor  scores  on  the  ASVAB  were  used.  Therefore,  in  the  interests  of  parsimony, 
scores  on  the  four  factors  were  used.  Hands-on  and  job  knowledge  criteria  were  analyzed 
separately. 

Time  in  service  has  a  major  influence  on  performance  scores  because  it  represents  training 
while  on  the  job.  Analyses  show  that,  as  the  recruit  learns  more,  the  learning  rate  decreases. 
Therefore,  in  each  regression,  TIS  and  TIS_SQ  were  first  entered  into  the  prediction  equation. 
Once  this  was  done,  the  regression  results  for  predictors  added  subsequently  indicated  what 
would  have  tuqipened  if  all  subjects  in  the  study  had  the  same  length  of  service. 

There  is  another  way  in  which  TIS  may  affect  the  prediction  of  criterion  scores.  As  time 
passes,  a  Marine’s  true  ASVAB  scores  may  change  somewhat  As  a  result,  the  regression 
coefficients  for  the  enlistment  factor  scores  may  decrease  as  TIS  increases.  For  each  of  the 
four  factors,  this  effea  was  allowed  for  by  adding  a  new  predictor  equal  to  the  product  of  the 
factor  score  with  TIS.  For  the  Verbal  factor  this  new  variable  was  called  T_VERBAL,  and  so  on. 

A  separate  analysis  was  performed  for  each  new  test  In  each  MOS,  the  four  ASVAB 
factors  and  then  the  new  test  were  entered  into  the  regressioa  The  resulting  residual  variance 
was  compared  with  that  using  only  the  four  factors.  (Residual  variance  equals  the  square  of  the 
standard  error  of  estimate.)  The  reduction  in  residual  variance  due  to  addition  of  the  new  test 
was  the  variance  explained  by  the  test  Such  calculations  automatically  incorporate  the  standard 
correction  used  in  adjusting  the  multiple  correlation  for  its  upward  bias  [5].  (Because  of  this 
correction,  when  the  reduction  in  residual  variance  is  smaller  than  that  expected  from  pure 
chance,  the  estimated  value  comes  out  negative.) 
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The  data  were  collected  in  a  sample  of  recniits  who  have  been  selected  using  the  ASVAB. 
Therefore,  validity  of  the  ASVAB  is  lower  in  this  selected  sample  than  in  the  national 
population.  Corrections  for  such  restriction  of  range  are  simple  for  the  quantities  of  interest  in 
this  paper.  First  only  the  ASVAB,  with  or  without  slope  changing  with  TIS,  is  used  in  the 
regression  equatioa  When  the  residual  variance  of  this  analysis  is  subtracted  from  100,  one 
obtains  the  variance  explained  by  the  ASVAB  in  the  national  population.  Then  one  new  test  is 
added  as  a  predictor,  and  the  additional  variance  explained  by  this  test  is  calculated.  According 
to  the  assumptions  used  in  range  correction,  this  variance  is  the  same  in  the  unselected  national 
population  as  in  the  selected  recruit  population. 

RESULTS 

Since  recruits  are  a  selected  sample,  their  ASVAB  scores  have  a  smaller  spread  than  in  the 
national  population.  The  degree  of  this  range  restriction  can  be  quantified  in  different  ways.  For 
the  purposes  of  this  paper,  the  relevant  ASVAB  “score”  is  the  prediaed  value  of  the  criterion 
variable,  using  the  enlistment  ASVAB  and  holding  TIS  constant  The  spread  of  this  score  in  the 
recruit  sample,  relative  to  the  national  population,  is  described  by  the  ratio  of  standard  deviations 
in  these  two  groups.  For  the  hands-on  criterion,  this  ratio  was  .634  in  MOS  0311,  .654  in 
MOS  0331,  .785  in  MOS  0341,  and  .696  in  MOS  0351.  For  job  knowledge  the  corresponding 
ratios  were  .526,  .562,  .686,  and  .542.  Thus,  MOS  0341  is  less  restricted  than  the  others. 

Table  1  shows  percentages  of  population  variance  explained  by  the  three  ways  of  using 
ASVAB  scores.  E_ASVAB  in  the  Predictor  column  means  that  the  enlistment  ASVAB  factors 
were  used  in  the  regression.  C_ASVAB  means  that  concurrent  ASVAB  factors  were  used. 
E.ASVAB'^  means  that  addition  of  T_VERBAL,  etc.,  allowed  regression  weights  to  change  with 
TIS.  Addition  of  this  interaction  term  turned  out  to  have  a  noticeable  effect  only  on  the  hands-on 
criterion  in  MOS  0341. 


Table  1.  Percent  variances  in  population  explained  by  enlistment  ASVAB 
(E_ASVAB).  by  enlistment  ASVAB  plus  interaction  wrth  TIS  (E_ASVAB*), 
and  by  concurrent  ASVAB  (C_ASVAB) 


Criterion 

Predictor 

MOS 

0311 

0331 

0341 

0351 

HOPT 

E  ASVAB 

44.7 

54.8 

41.2 

33.9 

HOPT 

E  ASVAB* 

44.8 

54.5 

44.5 

34.1 

HOPT 

C_ASVAB 

45.6 

55.7 

40.6 

39.8 

JKT 

E  ASVAB 

65.2 

59.2 

50.9 

55.6 

JKT 

E  ASVAB* 

65.1 

59.1 

52.2 

56.5 

JKT 

C  ASVAB 

67.8 

65.3 

56.9 

61.3 

Table  2  shows  incremental  percentages  of  explained  variance  due  to  the  four  new  predictors. 
Entries  in  the  Predictor  column  have  the  same  meanings  as  in  table  1.  Again,  in  most  cases,  the 
numbers  are  affected  very  little  by  allowing  ASVAB  regression  weights  to  change  with  TIS. 


Table  2.  Percent  variances  explained  by  new  tests  when  added  to 
enlistment  ASVAB  (E_ASVAB),  to  enlistment  ASVAB  plus  interaction 
with  TIS  (E_ASVAB*),  and  to  concurrent  ASVAB  (C_ASVAB) 


Criterion 

Predictor 

New  Test 

Firing 

Space 

Reasoning 

Objects 

MOS  031 1 :  Rifleman,  N  - 

>864 

HOPT 

E  ASVAB 

2.27 

0.52 

0.87 

1.26 

HOPT 

E  ASVAB* 

2.33 

0.55 

0.89 

1.27 

HOPT 

C_ASVAB 

2.09 

0.35 

0.46 

0.94 

JKT 

E  ASVAB 

0.13 

0.22 

1.49 

2.44 

JKT 

E  ASVAB* 

0.13 

0.22 

1.49 

2.41 

JKT 

C  ASVAB 

0.07 

0.02 

0.62 

1.48 

MOS  0331 :  Machinegunner, 

N-234 

HOPT 

E  ASVAB 

0.90 

2.56 

0.80 

0.70 

HOPT 

E  ASVAB* 

0.87 

2.64 

0.72 

0.63 

HOPT 

C_ASVAB 

1.14 

3.00 

1.29 

0.65 

JKT 

E  ASVAB 

0.11 

1.09 

0.26 

3.95 

JKT 

E  ASVAB* 

0.09 

1.20 

0.27 

4.11 

JKT 

C  ASVAB 

0.11 

0.99 

0.16 

2.76 

MOS  0341:  Moftarman,  N 

-223 

HOPT 

E  ASVAB 

0.56 

0.91 

0.76 

3.25 

HOPT 

E  ASVAB* 

0.49 

0.75 

0.79 

2.59 

HOPT 

C_ASVAB 

0.26 

0.68 

-0.08 

2.46 

JKT 

E  ASVAB 

1.36 

1.69 

4.69 

3.78 

JKT 

E  ASVAB* 

1.10 

1.49 

4.66 

3.41 

JKT 

C  ASVAB 

0.66 

1.24 

2.09 

2.62 

MOS  0351  ;Assaultman,  N 

-251 

HOPT 

E  ASVAB 

1.72 

0.88 

0.14 

1.19 

HOPT 

E  ASVAB* 

1.87 

0.97 

-0.01 

1.12 

HOPT 

C_ASVAB 

0.67 

0.27 

-0.14 

0.50 

JKT 

E  ASVAB 

0.03 

0.50 

0.45 

1.97 

JKT 

E  ASVAB* 

0.08 

0.52 

0.32 

1.84 

JKT 

C  ASVAB 

-0.15 

0.24 

-0.05 

0.70 
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The  primary  comparison  is  between  the  E_ASVAB*  and  C_ASVAB  rows  for  HOPT  within 
each  MOS.  Of  the  16  comparisons.  C_ASVAB  yields  smaller  increments  in  12  cases.  However, 
the  difference  is  usually  a  fraction  of  a  percem,  and  hence  much  less  important  than  the  differ¬ 
ence  between  one  MOS  and  another.  Differences  are  larger  for  JKT. 

The  additional  variance  explained  by  a  given  subtest  depends  on  the  criterion.  Video  Firing 
and  Space  Perception  contribute  more  to  HOPT  than  to  JKT;  the  opposite  is  true  of  Reasoning 
and  Assembling  Objects.  Variations  are  found  across  MOSs  as  well:  Video  Firing  makes  its 
largest  contribution  in  the  Rifleman  MOS.  Space  Perception  in  the  Machinegunner  MOS,  and 
Reasoning  and  Object  Assembly  in  the  Mortaiman  MOS.  Such  variations  make  it  very  difficult 
to  compare  the  tests. 

CONCLUSIONS 

In  conclusion,  as  a  matter  of  research  design,  a  concurrent  ASVAB  does  not  appear  to  be 
important  for  avoiding  an  upward  bias  in  evaluation  of  new  tests,  particulariy  if  hands-on 
performance  is  the  primary  criterion  variable.  The  difference  between  incremental  variances 
using  concurreiu  and  enlistment  ASVABs  is  minor  compared  to  variations  across  different 
criterion  variables  and  occupational  specialties.  In  addition,  it  should  be  remembered  that 
Marines  in  the  JPM  study  had  an  incentive  to  do  well  on  the  concurrent  ASVAB — a  chance  to 
increase  their  scores  of  record.  Without  such  an  incentive  to  improve  motivation,  a  concurrent 
ASVAB  may  yield  misleading  results. 

The  changes  in  incremental  variance  across  different  MOS  and  from  HOPT  to  JKT  should 
be  seen  as  a  danger  sigiuil.  The  research  design  for  the  ECAT  validity  study  of  new  com¬ 
puterized  tests  is  given  by  Wolfe  [6].  This  study  covers  31  differem  occupations.  The  criterion 
variables  include  hands-on  performance,  supervisor  ratings,  job  knowledge  tests,  performance  on 
simulators  and  in  laboratories  (when  available),  and  training  school  grades.  Different  criteria 
will  be  available  in  different  occupations.  If  results  of  this  study  show  the  kinds  of  variations 
found  in  table  2,  it  will  be  very  difficult  to  summarize  them  and  to  conclude  that  one  test  is  more 
useful  than  another. 
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